Video Monitoring System, Video Monitoring Method, and Video Monitoring System Building Method

ABSTRACT

It is an objective of the present invention to reduce processing loads for video analysis utilizing information throughout the facility. A video monitoring system according to the present invention simulates a flow of a moving object within a video captured by multiple monitoring cameras, calculates a parameter correlated with a processing load for movement analysis of the moving object according to the simulation result, and specifies a processing scheme that is capable of reducing the processing load according to a correspondence relationship between the parameter and the simulation result (refer to FIG.  2 ).

CLAIM OF PRIORITY

The present application claims priority from Japanese patent applicationJP 2013-178524 filed on Aug. 29, 2013, the content of which is herebyincorporated by reference into this application.

BACKGROUND

1. Technical Field

The present invention relates to a technique for monitoring monitoredobjects using videos.

2. Background Art

In recent years, there are increasing needs for identifying human flowsin facilities to efficiently detect congestion conditions or occurrenceof troubles. Regarding such needs, there is a system that identifieshuman flows in facilities using monitoring cameras installed inlocations where people gather such as stores or airports, therebydetecting congestion conditions or occurrence of troubles. However, insuch systems, only the information within the field of view of themonitoring cameras is available. Thus it is difficult to identify theconditions throughout the facility.

Regarding the above-described problem, Patent Document 1 listed belowdescribes a technique that estimates, according to the informationacquired from the monitoring cameras, moving paths of persons at blindspots of the cameras. In addition, Patent Document 2 listed belowdescribes, regarding video monitoring, a technique that decreasesprocessing loads by switching analysis processes. Non-Patent Documents 1and 2 listed below disclose techniques, as techniques regarding videomonitoring, that extract movements from videos.

RELATED ART DOCUMENTS Patent Documents

-   Patent Document 1: WO 2007/026744-   Patent Document 2: JP Patent Publication (Kokai) 2007-264706 A

Non-Patent Documents

-   Non-Patent Document 1: Dirk Helbing and Peter Molnar, “Social Force    Model for Pedestrian Dynamics”, Physical Review E, vol. 51, no. 5,    pp. 4282-4286, 1995.-   Non-Patent Document 2: S. Baker and I. Matthews, “Lucas-kanade 20    years on: A unifying framework”, International Journal of Computer    Vision, vol. 53, no. 3, 2004.

SUMMARY

When estimating moving paths of persons, it is necessary to extractinformation required to estimate the moving paths from the video ofmonitoring cameras. At this time, there is a technical problem regardingthe processing load to extract information that is required to estimatethe moving paths from multiple monitoring cameras. For example, a caseis assumed where videos of multiple monitoring cameras are aggregated onthe same server and information is extracted on the server. In such acase, the processing load on the server becomes higher as the number ofmonitoring cameras processed by one server is increased. Accordingly,the time span to complete information extraction for all cameras couldbe longer than the time span to input the camera images. In this case,it is impossible to extract information on a real-time basis. Thus it isimpossible to identify the condition of the facility on a real-timebasis.

Regarding the above-described problem, in Patent Document 2 above, theprocessing load for analysis is decreased by determining the congestionlevel according to the camera images and by switching the analysisprocess according to the determination result. However, in thistechnique, the analysis process is switched using the informationavailable from a single camera. Therefore, it is difficult to reduce theprocessing load utilizing the information throughout the facility.

The present invention is made in order to solve the above-mentionedproblem. It is an objective of the present invention to reduceprocessing loads for video analysis utilizing information throughout thefacility.

A video monitoring system according to the present invention simulates aflow of a moving object within a video captured by multiple monitoringcameras, calculates a parameter correlated with a processing load formovement analysis of the moving object according to the simulationresult, and specifies a processing scheme that is capable of reducingthe processing load according to a correspondence relationship betweenthe parameter and the simulation result.

With the video monitoring system according to the present invention, itis possible to identify a condition throughout the facility by asimulation using multiple cameras, thereby reducing a processing loadfor video analysis based on the result thereof.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic diagram of a video monitoring system 100 accordingto an embodiment 1.

FIG. 2 is a functional diagram of a video monitoring device 200 includedin the video monitoring system 100 according to the embodiment 1.

FIG. 3 is a functional block diagram showing a configuration example ofa movement feature extractor 202, a simulator 203, and a control signalgenerator 205.

FIG. 4 is a diagram exemplifying a timing for a switcher 302 to switchbetween a test period and a normal period.

FIG. 5 is a diagram showing a configuration and a data example of a testpattern generation table 500 stored in a test control signal generator311.

FIG. 6 is a functional block diagram showing a configuration example ofa normal movement feature extractor 303.

FIG. 7 is a functional block diagram showing a configuration example ofa feature converter 602.

FIG. 8 is a flowchart of a process for determining a control signal 211in a test period 401.

FIG. 9 is a diagram showing a hardware configuration example of thevideo monitoring device 200.

FIG. 10 is a diagram showing a network configuration example of thevideo monitoring system 100.

FIG. 11 is a functional block diagram of the video monitoring device 200according to an embodiment 2.

FIG. 12 is a functional block diagram showing a configuration example ofthe control signal generator 205 in the embodiment 2.

FIG. 13 is a diagram showing a configuration and a data example of aconfiguration pattern correspondence table 1300 included in a conditionassociation unit 1201.

FIG. 14 is a functional block diagram of the video monitoring device 200according to an embodiment 3.

FIG. 15 shows an example of a switching signal generated by a testtiming generator 301.

FIG. 16 is a functional block diagram of the video monitoring device 200according to an embodiment 4.

FIG. 17 is a flowchart showing a sequence for building the videomonitoring system 100.

FIG. 18 is a functional block diagram of the video monitoring device 200according to an embodiment 6.

DETAILED DESCRIPTION OF THE EMBODIMENT(S) Embodiment 1

FIG. 1 is a schematic diagram of a video monitoring system 100 accordingto an embodiment 1 of the present invention. In FIG. 1, monitoredobjects (movable bodies) (101, 102, 103) existing in a monitored areaare monitored by monitoring cameras (104, 105). Each of the monitoringcameras has a limited field of view (106) for capturing videos. Thusthere is a blind area 108 that is not monitored by any one of thecameras.

Firstly, the video monitoring system 100 extracts movement features(110, 111, 112) of the monitored object from the video captured by eachof the cameras. At this time, the movement features (110, 112) withinthe field of view of each monitoring camera are extracted by performingimage processing with respect to the video of the monitoring camera. Theextracted movement features are converted into movement features (113,114) of overhead view. The movement feature 111 of the blind area cannotbe extracted because it is not monitored by the monitoring cameras. Thusonly the movement feature within the monitored area 115 is available.The video monitoring system 100 then performs a simulation using theacquired movement features to estimate movement features of the objectthroughout the facility. This enables acquiring movement features ofmonitored objects throughout the facility.

In the embodiment 1, as an example of estimating movement features ofmonitored objects throughout the facility, a method will be described inwhich movement features are more precisely estimated than PatentDocument 1 by simulating flows of monitored objects using informationacquired from sensors in addition to information acquired from videos ofmonitoring cameras. Examples of information available from sensors maybe such as, counted results for passing number of monitored objects byinfrared sensors, or estimated results for number of monitored objectsby load sensors. When performing the simulations, methods such asdescribed in Non-Patent Document 1 are used to estimate the movementmodel of monitored objects, and the movement directions of the monitoredobjects are simulated according to the movement models. This enablesestimating the flows of monitored objects more particularly than PatentDocument 1.

FIG. 2 is a functional diagram of a video monitoring device 200 includedin the video monitoring system 100 according to the embodiment 1. Thevideo monitoring device 200 includes a movement feature extractor 202, asimulator 203, a simulation result viewer 204, a control signalgenerator 205, and a format converter 208.

The movement feature extractor 202 receives a camera image 201 capturedby each of multiple monitoring cameras. The movement feature extractor202 extracts a movement feature 209 from each of the camera images 201.The movement feature is information describing movement paths of movingobjects captured in monitoring camera images. For example, the movementfeature is described by vectors in which coordinate locations of movingobjects are aligned for each of time. According to a control signal 211given from the control signal generator 205, the movement featureextractor 202 switches processing schemes that are used when extractingmovement features. Details of the switching will be described later. Theextracted movement feature 209 is inputted into the simulator 203.

The format converter 208 receives sensor information 207 describingphysical states detected by each sensor. The format converter 208converts the sensor information 207 into formats that can be handled bythe simulator 203. For example, if the sensor information 207 describescounted results for the number of monitored objects by infrared sensors,location information of the sensor is attached to the counted result ofmonitored objects, and then outputted. The sensor information 207 inwhich the format converter 208 converts its format is referred to assensor feature 210.

The simulator 203 receives the movement feature 209 and the sensorfeature 210. The simulator 203 performs a simulation using the movementfeature 209 and the sensor feature 210 to calculate movement features ofmonitored objects throughout the facility. The movement feature ofmonitored objects throughout the facility acquired by this simulation isreferred to as a simulation result 212. The simulation result 212 isinputted into the simulation result viewer 204 and the control signalgenerator 205.

The simulation result viewer 204 performs processes such as projectingthe simulation result 212 onto the map of the facility, and displays iton such as displays.

The control signal generator 205 receives the simulation result 212 andan objective processing load 206. The control signal generator 205generates the control signal 211 for switching the processing schemeused when the movement feature extractor 202 extracts the movementfeature 209. The control signal generator 205 then outputs the controlsignal 211 into the movement feature extractor 202. The control signalgenerator 205 generates the control signal 211 so that the processingload of the video monitoring device 200 will be decreased. By generatingthe control signal 211 reducing the processing load according to thesimulation result 212, the analysis process is planned to be completedon a real time basis. By generating the control signal 211 according tothe simulation result 212 that is generated using the camera image 201acquired from multiple cameras, it is possible to adjust the processingload in the light of the condition throughout the facility. Therefore,it is possible to perform a control more preferable than adjusting theprocessing load using a video acquired from a single camera.

In the embodiment 1, the control signal is generated so that theprocessing load of the process extracting movement features for allmonitoring cameras is restricted within the objective processing load206. If an objective is to acquire the simulation result on a real timebasis, the objective processing load 206 may be configured so that theprocess of extracting features for all cameras will be completed withinthe time interval at which the videos are inputted from the cameras. Theobjective processing load 206 may be configured as a reference ofprocessing duration required for completing the calculation, forexample.

FIG. 3 is a functional block diagram showing a configuration example ofthe movement feature extractor 202, the simulator 203, and the controlsignal generator 205. During operations, these functional units switchbetween a test period for performing simulations using the camera image201 in order to determine the control signal 211 and a normal period forextracting movement features according to the determined control signal211.

The movement feature extractor 202 includes a switcher 302, a normalmovement feature extractor 303, a high load movement feature extractor305, a test movement feature extractor 307, and a processing loadcalculator 310. The simulator 203 includes a normal simulator 304, ahigh load simulator 306, and a test simulator 308. The control signalgenerator 205 includes a test timing generator 301, a matching degreecalculator 309, a test control signal generator 311, and a controlsignal determinator 312.

The test timing generator 301 generates a timing for switching betweenthe test period and the normal period, and outputs a signal for theswitching. Examples of timing for switching will be described with FIG.4 later.

The switcher 302 switches between the test period and the normal periodaccording to the switching signal inputted from the test timinggenerator 301. If the switching signal generated by the test timinggenerator 301 indicates the normal period, the normal simulator 304performs a simulation and outputs the acquired simulation result intothe simulation result viewer 204. If the switching signal generated bythe test timing generator 301 indicates the test period, the cameraimage 201 is inputted into the high load movement feature extractor 305and into the test movement feature extractor 307. Processes of theseextractors will be described later.

The high load movement feature extractor 305 extracts movement featuresusing parameters that cause the highest processing load amongconfigurable parameters. The extracted movement feature is inputted intothe high load simulator 306. The high load simulator 306 performs asimulation using the received movement features and outputs thesimulation result. The simulation result acquired by this process is aresult of a simulation using features acquired from the high loadfeature extraction process. Thus it can be assumed that such simulationresult has a high precision. Hereinafter, this simulation result will bereferred to as a high precision simulation result 314. The highprecision simulation result 314 is inputted into the matching degreecalculator 309.

The test control signal generator 311 sequentially selects controlsignal patterns described by a test pattern generation table 500exemplified with FIG. 5 later, and generates control signals for testsaccording to those patterns.

The test movement feature extractor 307 extracts features according tothe control signal for tests generated by the test control signalgenerator 311. The extracted movement feature is inputted into the testsimulator 308. The test simulator 308 performs a simulation using thereceived movement feature, and outputs the simulation result. Thisacquires a simulation result corresponding to the control signal fortests. This simulation result will be referred to as a test simulationresult 315. The test simulation result 315 is inputted into the matchingdegree calculator 309.

The processing load calculator 310 calculates processing loads when thetest movement feature extractor 307 extracts movement features accordingto the control signal for tests. The calculated processing load isinputted into the control signal determinator 312.

The matching degree calculator 309 calculates a matching degree betweenthe high precision simulation result 314 and the test simulation results315 corresponding to each of the control signals for tests. The matchingdegree may be calculated by, for example, comparing a matching degreebetween a histogram in the moving direction of monitored objects in thehigh precision simulation result 314 at a certain time and a histogramin the moving direction of monitored objects in the test simulationresult 315 at the same time, using such as Bhattacharyya distance. Thehigher the matching degree is, the more precise the test simulationresult 315 is. The calculated matching degree is inputted into thecontrol signal determinator 312.

The control signal determinator 312 determines the preferable controlsignal 211 using the matching degree corresponding to each of thecontrol signals for tests and the processing load when extractingfeatures. The sequence for determination will be described using FIG. 5later. The normal movement feature extractor 303 extracts movementfeatures according to the control signal 211 determined by the controlsignal determinator 312. This acquires simulation results with highprecision on a real time basis while reducing processing loads.

FIG. 4 is a diagram exemplifying a timing for the switcher 302 to switchbetween the test period and the normal period. The video monitoringsystem 100 performs a test for configuring the control signal 211 duringthe test period 401 according to the above-described sequence, and worksduring the normal period 402 using the control signal 211. The switcher302 generates switching signals so that the test period 401 and thenormal period 402 are repeated.

FIG. 5 is a diagram showing a configuration and a data example of thetest pattern generation table 500 stored in the test control signalgenerator 311. The test pattern generation table 500 is data describingcombination patterns of the camera images 201 to which processingschemes for reducing processing loads when extracting movement featuresare applied. For example, the test pattern of the first line in FIG. 5shows that: regarding the camera image 201 acquired from camera A andcamera C, a processing scheme is applied that uses a parameter amongconfigurable parameters causing a processing load which is 50% of theprocessing scheme using the highest processing load. For the sake ofconvenience of description, FIG. 5 also shows processing resultsacquired by using test patterns described by the test pattern generationtable 500.

“matching degree” shown in FIG. 5 is a matching degree between the testsimulation result 315 corresponding to each of the test patterns and thehigh precision simulation result 314. It can be supposed that as theprocessing load used by the processing scheme becomes higher, thematching degree also becomes higher. Thus the matching degree has acorrelation with the processing loads of each of the movement featureextractors. “ratio of processing load with respect to objectiveprocessing load” shown in FIG. 5 shows ratios of each test pattern withrespect to the objective processing load 206. If this value is at orbelow 100%, it shows that the objective processing load 206 is achieved(e.g. the process is completed on a real time basis). If there aremultiple test patterns achieving the objective processing load 206, thecontrol signal determinator 312 determines the control signal 211 amongthem achieving the highest matching degree.

In the above-described sequence, the processing load during the testperiod 401 may increase to disable real-time processing. Even in suchcases, the simulation result viewer 204 can show some degree ofinformation by presenting the high precision simulation result 314,though it includes some delay from the real time simulation result. Inaddition, overall processing loads can be decreased by securing thenormal period 402 sufficiently longer than the test period 401.

In the above-described sequence, the control signal determinator 312selects a test pattern with the highest matching degree among the testpatterns below the objective processing load 206. It is also possible toselect a test pattern as the control signal 211 with the lowestprocessing load among the test pattern achieving the objective matchingdegree.

In the timing example shown in FIG. 4, the test timing generator 301switches between the test period 401 and the normal period 402 at aconstant interval. It is also possible to measure the processing load inthe normal period 402 in advance, and to perform the switching into thetest period 401 when the processing load excesses a predetermined value.

In the above-described configuration example, it is possible to recordthe camera image 201 and to extract movement features based on therecorded video. In this case, the high load movement feature extractor305 and the test movement feature extractor 307 may perform processingat different timings using the same video. By shifting the timings atwhich the high load movement feature extractor 305 and the test movementfeature extractor 307 perform processes from each other, it is possibleto restrict temporary increase in processing load during the test period401.

FIG. 6 is a functional block diagram showing a configuration example ofthe normal movement feature extractor 303. The high load movementfeature extractor 305 and the test movement feature extractor 307 alsoinclude similar configurations. The camera image 201 is inputted into amovement vector calculator 601. The movement vector calculator 601extracts movement vectors from the camera image 201 using, for example,the scheme described in Non-Patent Document 2. The extracted movementvector is inputted into a feature converter 602. The feature converter602 converts the movement feature extracted by the movement vectorcalculator 601 into the movement feature 209 in which simulations can beperformed. The control signal 211 for configuring parameters used by themovement vector calculator 601 is inputted into a parameter configurator603. According to the control signal 211, the parameter configurator 603changes processing parameters used by the movement vector calculator 601when extracting the movement vectors. This parameter is a parameter thatinfluences on the processing load of the movement vector calculator 601.For example, this parameter is a parameter that specifies a processingfrequency for calculating the movement vector or that specifies areasfor the processing. If the control signal 211 instructs to decrease theprocessing load for calculating the movement vector by 50%, theprocessing frequency parameter is reduced to half, so that the featureis extracted once every two times the camera image is inputted. Thisreduces the processing load by 50%. The parameter configurator 603stores a correspondence table between the control signal 211 and theprocessing parameters in advance, and converts the control signal 211into parameters of feature extraction process using the correspondencetable. The movement vector calculator 601 calculates movement vectorsusing the parameters.

FIG. 7 is a functional block diagram showing a configuration example ofthe feature converter 602. The example shows that a movement vector 701extracted by the movement calculator 601 is described by a 2 dimensionmovement vector corresponding to image coordinates. The simulator 203uses movement features seen from an overhead view as the movementfeatures of monitored objects. The feature converter 602 includes avector unifier 702, a 3 dimension location estimator 703, and acoordinate converter 705.

The feature includes a 2 dimension location on the image coordinate anda movement feature describing movements. The vector unifier 702 unifiesthese vectors assuming that adjacent vectors among the movement vectors701 are highly likely to be vectors of the same object. For example,commonly known Mean-Shift clustering technique may be used to unify thevectors. The 3 dimension location estimator 703 converts the 2 dimensionlocation on the image coordinate where the feature has been extractedinto a 3 dimension location in the real space. This conversion can beeasily performed if the angle of field of cameras, the focal length, theheight from the ground, the angle of the camera with respect to theground, and the height of the feature in the real space are known. Theangle of view of cameras, the focal length, the height from the ground,and the angle of the camera with respect to the ground are configured inadvance. The height of feature extraction point in the real space can beestimated by such as the method below.

The height of feature in the real space can be estimated using therelationship between the monitored object and the ground, for example.If humans are monitored objects, human regions are extracted using humanextracting process. Assuming that the extracted human is standing on theground, the height of the human's foot matches with the height of theground. By assuming that the body height of the extracted human is acertain value, it is possible to estimate the height of featuresincluded in the human region. Template matching or the like may be usedas the human extracting process. The 3 dimension location estimator 703performs the above-described process with respect to each of elements inthe 2 dimension movement vector of the feature, thereby converting thefeature into the 3 dimension movement vector 704.

The coordinate converter 705 performs coordinate translation from cameracoordinate into overhead view coordinate. In other words, the 3dimension movement vector 704 converted into 3 dimension location isconverted into the 2 dimensional movement vector 706 seen from overheadviewpoints. The aforementioned process can convert the feature of themovement vector 701 into features seen from overhead viewpoints.

FIG. 8 is a flowchart of a process for determining the control signal211 in the test period 401. Hereinafter, each step in FIG. 8 will bedescribed.

(FIG. 8: step S801)

The control signal generator 205 sets the objective processing load 206.The objective processing load 206 may be, for example, specified by auser of the video monitoring system 100, or may be acquired throughappropriate communication networks or storage media.

(FIG. 8: steps S802-S803)

The video monitoring system 100 performs steps S803-S808 below withrespect to all test patterns described in the test pattern generationtable 500 (S802). The video monitoring system 100 further performs stepsS804-S805 below with respect to all cameras (S803).

(FIG. 8: steps S804-S805)

The high load movement feature extractor 305 extracts movement featuresfrom the camera image 201 using the parameter causing the highest load(S804). The test movement feature extractor 307 extracts movementfeatures from the camera image 201 according to each test patterndescribed by the test pattern generation table 500 (S805).

(FIG. 8: steps S806-S808)

The processing load calculator 310 calculates the processing load whenthe test movement feature extractor 307 extracts movement features(S806). The simulator 203 calculates the simulation results of the highload simulator 306 and the test simulator 308 respectively (S807). Thematching degree calculator 309 calculates the matching degree betweenthe high load simulation result and the test simulation result (S808).

(FIG. 8: step S809)

The control signal determinator 312 determines the control signal 211 onthe basis of the processing load calculated in step S806 and of thematching degree calculated in step S808 according to the sequencedescribed in FIG. 5.

FIG. 9 is a diagram showing a hardware configuration example of thevideo monitoring device 200. The video monitoring device 200 includes aprocessor 901, a storage device 902, and an I/O device 903. Theprocessor 901 and the storage device 902 are connected to a userinterface 904, a display device 905, a monitoring camera 906, and arecording device 907 through the I/O device 903.

The processor 901 calls and executes necessary processes frominstructions 917 stored in the storage device 902. The instructions 917are programs describing processes corresponding to the movement featureextractor 202, the simulator 203, the control signal generator 205, andthe simulation result viewer 204.

Camera install state information 912 describes configuration parametersof cameras that are used when extracting movement features from thecamera image 201. An objective processing load 916 corresponds to theobjective processing load 206. These pieces of data are inputted throughthe user interface 904, and are stored in the storage device 902. Acontrol signal 914 corresponds to the control signal 211 stored in thestorage device 902.

The movement feature extractor 202 acquires the camera image 201captured by the monitoring camera 906 or the camera image 201 recordedby the recording device 907, reads out the camera install stateinformation 912 and the control signal 914 from the storage device 902,and extracts the movement feature 913 using them to store it in thestorage device 902. The simulator 203 reads out the movement feature913, performs a simulation, and store a simulation result as thesimulation result 915 in the storage device 902. The control signalgenerator 205 reads out the simulation result 915 and the objectiveprocessing load 916, and generates the control signal 914 using them.The simulation result viewer 204 reads out the simulation result 915,and converts it into video formats that can be displayed on the displaydevice 905. The display device 905 displays the video.

In the configuration example shown in FIG. 9, other data required forperforming simulations may be stored in the storage device 902. Forexample, map information of facilities required for simulation, timeinformation or weather information for performing detailed simulations,and the like may be stored.

FIG. 10 is a diagram showing a network configuration example of thevideo monitoring system 100. In FIG. 10, monitoring cameras 1003 areconnected to the video monitoring device 200 and the recording device907 through a network 1004. The display device 905 is connected to thevideo monitoring device 200 through a network 1006. A client terminal1007 includes the user interface 904 for inputting the objectiveprocessing load 916, and is connected to the video monitoring device 200through a network 1009. Videos from the monitoring cameras 1003 and therecording device 907 are inputted into the video monitoring device 200through the network 1004. The objective processing load 916 configuredby the client terminal 1007 is inputted into the video monitoring device200 through the network 1009. The simulation result 915 calculated bythe video monitoring device 200 is inputted into the display device 905through the network 1006. The display device 905 displays the result.

In FIGS. 9 and 10, it is assumed that the video monitoring device 200extracts movement features from the camera image 201. Each of cameras inthe monitoring cameras 1003 may perform it. In addition, it is assumedthat the client terminal 1007 performs the objective processing loadconfiguring process 1008. The process may be performed by directlyconnecting I/O terminals to the video monitoring device 200.

In FIGS. 9 and 10, when storing the movement feature 913, not onlyvector data describing the movement features but also cameraconfiguration parameters or processing parameters used in movementanalysis may be stored. In addition, the processing scheme of thesimulator 203 may be switched using this information.

In the embodiment 1, the simulator 203 performs simulations using thesensor information 207. If it is possible to execute simulations withsufficient precision even without the sensor information 207, it is notnecessary to use the sensor information 207.

Embodiment 1 Summary

As discussed thus far, the monitoring system 100 according to theembodiment 1 employs, as the control signal 211, a test pattern that isthe closest to the high precision simulation result 314 among testpatterns corresponding to each of the test simulation result 315. Sinceeach simulation result is acquired using multiple of the camera images201, it is possible to determine the control signal 211 consideringconditions throughout the facility. This enables completing the featureextraction for all cameras on a real time basis, and acquiring thesimulation result on a real time basis. In addition, even if theprocessor 901 has a sufficient processing performance, it is possible todecrease the usage rate of the processor 901 to reduce electric powerconsumption of the overall system by decreasing the processing loadusing the embodiment 1.

Embodiment 2

In the embodiment 1, a configuration example is described where thecontrol signal 211 is selected from the test patterns described in thetest pattern generation table 500. In an embodiment 2 of the presentinvention, a configuration example will be described where movements ofmonitored objects are predicted by simulation, the processing load forextracting the movement features is adjusted according to the predictedmovement, and the analysis accuracy is optimized according to themovements of monitored objects. Other configurations are approximatelythe same as the embodiment 1. Thus hereinafter differences will bemainly described.

FIG. 11 is a functional block diagram of the video monitoring device 200according to the embodiment 2. In the embodiment 2, map information 1101is used to identify camera locations corresponding to movements ofmonitored objects. The map information 1101 is information showinglayouts of the facility where the video monitoring system 100 isinstalled. The map information 1101 describes sizes/locations/types ofobjects disposed in the facility by coordinate values and attributevalues in a common coordinate system. Install locations or installstates of cameras may be managed in the map information 1101. The mapinformation 1101 is stored in the storage device 902 in advance, forexample.

The simulator 203, the simulation result viewer 204, and the controlsignal generator 205 acquire the map information 1101. The simulator 203performs simulations using the map information 1101 in addition to themovement feature 209 and the sensor feature 210. For example, if the mapinformation 1101 includes information of walls, the location of walls isdesignated as simulation conditions. This enables precisely simulatingthe flows of monitored objects. The simulation result viewer 204superimposes the simulation result on a visualized result of the layoutinformation included in the map information 1101. This enables plainlypresent the simulation result. The method by which the control signalgenerator 205 uses the map information 1101 will be described later.

FIG. 12 is a functional block diagram showing a configuration example ofthe control signal generator 205 in the embodiment 2. The control signalgenerator 205 includes, in addition to or instead of the configurationdescribed in the embodiment 1, a condition association unit 1201 and acontrol signal determinator 1202.

The simulation result 212 and the map information 1101 are inputted intothe condition association unit 1201. The condition association unit 1201associates, using the map information 1101, the simulation result 212with configuration patterns described later. Details will be describedlater.

A configuration pattern 1204, a processing load 1203 when the movementfeature extractor 202 extracts the movement features, and the objectiveprocessing load 206 are inputted into the control signal determinator1202. The control signal determinator 1202 determines the control signal211 using the configuration pattern 1204 and the processing load 1203.Details will be described later.

FIG. 13 is a diagram showing a configuration and a data example of aconfiguration pattern correspondence table 1300 included in thecondition association unit 1201. The configuration patterncorrespondence table 1300 describes multiple configuration patterns thatare determined by combinations of density of monitored objects atcertain points on the map information 1101 and the processing loads ofeach camera. Priorities are set for each of configuration patterns.According to the movement of monitored object acquired from thesimulation result 212 and the map information 1101, the conditionassociation unit 1201 calculates densities of monitored objects atpoints specified by each of the configuration patterns, selects allconfiguration patterns corresponding to the calculation result, andinputs the selected configuration patterns into the control signaldeterminator 1202 as the configuration pattern 1204.

If the density of monitored objects at a point A is at or below apredetermined value, it can be assumed that decreasing the analysisaccuracy does not significantly influence on the simulation resultregarding a camera A capturing the point A. Thus in the configurationpattern 1 in FIG. 13, the movement extraction process load of the cameraA is decreased down to 50%. It can be identified according to the mapinformation 1101 that the camera A is shooting the point A. If it isimpossible to decrease the overall processing load below the objectiveprocessing load 206 only by applying a single configuration pattern,processing loads of other cameras will be decreased according to thesequence below.

Firstly, the control signal determinator 1202 sets a configurationpattern having the highest priority as the control signal 211. Theprocessing load calculator 310 measures the processing load 1203according to this configuration pattern. If the measured processing load1203 is below the objective processing load 206, this is determined asthe control signal 211. If the processing load 1203 is higher than theobjective processing load 206, and if there is a camera with aprocessing load lower than the current control signal 211 amongconfiguration patterns having the next highest priority, a new controlsignal 211 is generated using the processing load in the configurationpattern having the next highest priority for that camera. In the dataexample shown in FIG. 13, the processing load of camera B in theconfiguration pattern 2 is lower than the processing load of camera B inthe configuration pattern 1. Thus the configuration pattern 2 isemployed for camera B. As a result, the control signal 211 is: cameraA=50%, camera B=50%, and camera C=100%. The control signal determinator1202 repeats the same sequence until the processing load 1203 becomesbelow the objective processing load 206 or all configuration patternsare combined.

The processing load 1203 may not become below the objective processingload 206 even by combining all configuration patterns. In such cases, apredetermined configuration pattern that surely achieves the objectiveprocessing load 206 may be prepared in advance, such as theconfiguration pattern 4 shown in FIG. 13, and such configuration patterncan be employed as the control signal 211. It is not necessary todecrease the processing loads for all cameras. It is sufficient todecrease the processing load of each of movement feature extractorstotally.

The configuration pattern correspondence table 1300 may be createdaccording to the processing load 1203 that is measured using such asrecorded videos in advance. For example, an experiment is performed tocalculate movement features of recorded videos, configuration patternsare determined that do not deteriorate accuracy of simulation resultseven with reduced processing loads, and such configuration patterns maybe described in the configuration pattern correspondence table 1300.

If the density of monitored objects becomes higher, the processing loadsof each movement feature extractor may also become higher. In otherwords, “condition” in the configuration pattern correspondence table1300 has a correlation with the processing load of each movement featureextractor. Parameters such as moving speed or complexity of movementother than density of monitored objects may have similar roles. Thusthese parameters may be used instead of the density.

Embodiment 2 Summary

As discussed thus far, the video monitoring system 100 according to theembodiment 2 can adjust the processing load for extracting movementfeatures in accordance with the movement of monitored objects, byassociating the simulation result 212 with the configuration patterns.This eliminates needs for the test period 401 for determining thecontrol signal 211, thereby acquiring simulation results whilemaintaining a constant processing load.

In the example shown in FIG. 13, if the density of monitored objects atpoint A is decreased, it is assumed that the processing load of camera Afor shooting point A may also be reduced. As another operationalexample, the destination of monitored object may be predicted bysimulation, thereby previously reducing the processing load of thecamera at the destination according to the prediction result.

Embodiment 3

FIG. 14 is a functional block diagram of the video monitoring device 200according to an embodiment 3 of the present invention. In the embodiment3, under the configuration described in the embodiment 1, aconfiguration example will be described where the processing load isgradually adjusted while switching between the normal period 402 and thetest period 401. Other configurations are similar to those of theembodiment 1. Thus hereinafter differences will be mainly described.

In the embodiment 3, the control signal generator 205 includes aswitcher 1401, a process result storage 1403, a control signal generator1404, and a matching degree storage 1406. Other functional units are thesame as those of the embodiment 1. Hereinafter, each of the functionalunits and their cooperation will be described.

The switching signal generated by the test timing generator 301 isinputted into the switcher 1401. If the switching signal indicates thenormal period 402, the process in the normal period 402 is performed. Ifthe switching signal indicates the test period 401, the process in thetest period 401 is performed.

In the normal period 402, the simulation result 212 is inputted into theprocess result storage 1403. The process result storage 1403 storessimulation results. The control signal generator 1404 outputs thecontrol signal 211 determined by the control signal determinator 312.

In the test period 401, the simulation result 212 is inputted into thematching degree calculator 309. The simulation results of the normalperiod 402 stored in the process result storage 1403 and the simulationresults of the test period 401 are inputted into the matching degreecalculator 309. The matching degree calculator 309 compares thesimulation result of the normal period 402 with the simulation result ofthe test period 401, and calculates the matching degree between them.The sequence for calculating the matching degree is the same as that ofthe embodiment 1. The calculated matching degree is associated with thecurrent control signal for tests and is stored in the matching degreestorage 1406. The test control signal generator 311 generates thecontrol signal for tests as in the embodiment 1. The test timinggenerator 301 determines which test pattern the test control signalgenerator 311 will output. When instructed to determine the controlsignal 211 by the test timing generator 301, the control signaldeterminator 312 determines the control signal 211 using the matchingdegree stored in the matching degree storage 1406. After determining thecontrol signal 211, the control signal determinator 312 deletes allmatching degrees stored in the matching degree storage 1406.

FIG. 15 shows an example of a switching signal generated by the testtiming generator 301. This example shows a signal for the switcher 1401,a signal for the test control signal generator 311, and a signal for thecontrol signal determinator 312. The test timing generator 301 comparesthe processing load 1203 with the objective processing load 206. In aperiod 1507 where the processing load 1203 is larger than the objectiveprocessing load 206, it is necessary to reduce the processing load 1203.Thus a signal to instruct to repeat the normal period 402 and the testperiod 401 alternatively is outputted as the signal for the switcher1401. In addition, a signal to sequentially switch the test patterns(test patterns 1-6) used in the test period 401 is outputted as thesignal for the test control signal generator 311. The test patterns usedin the test period 401 are selected among all test patterns where thesum of preset values of processing loads is lower than that of thecurrent control signal 211. After the test period 401, the test timinggenerator 301 outputs a signal to instruct to determine the controlsignal 211 as the signal for the control signal determinator 312. Afterdetermining the control signal 211, in the period where the processingload 1203 is at or lower than the objective processing load 206, thetest timing generator 301 outputs a signal to perform the normal period402 toward the switcher 1401.

By generating test timings as above, when the processing load 1203becomes larger than the objective processing load 206, it is possible toselect the control signal 211 with low processing loads to reduce theprocessing load.

In FIG. 15, an example is shown where the test timing generator 301instructs to determine the control signal 211 when all selected testpatterns have been processed. It is also possible to begin from the testpattern which is close to the sum of processing load of the currentcontrol signal 211, and to instruct to determine the control signal 211when a predetermined number of the test patterns have been completed.

In the examples above, the sequence for reducing the processing load1203 is described. If the processing load 1203 is much less than theobjective processing load 206, the processing load 1203 may beincreased. In order to increase the processing load 1203, some of testpatterns having large sum value of processing loads are tested amongtest patterns having sum value of processing loads close to the currentcontrol signal 211, and then the test pattern having the lowest matchingdegree is determined as the control signal 211. By repeating thisprocess until the processing load 1203 becomes close to the objectiveprocessing load 206, it is possible to acquire the simulation resultwhile maintaining the processing load close to the objective processingload 206.

In the examples above, an operational example is described where thetest period 401 and the normal period 402 are repeated intermittently.When the test period 401 should not be performed, such as when troubleshave occurred in the facility, an instruction to stop the test period401 and to perform the normal period 402 only may be sent to the videomonitoring device 200, and the video monitoring device 200 may workaccording to the instruction.

Embodiment 4

FIG. 16 is a functional block diagram of the video monitoring device 200according to an embodiment 4 of the present invention. In the embodiment4, a configuration example will be described where functional unitsother than the movement feature extractor 202 are controlled using thecontrol signal 211. The example below adjusts configuration parametersof monitoring cameras using the control signal 211. Other configurationsare similar to those of the embodiments 1-3. Thus hereinafterdifferences will be mainly described.

A monitoring camera 1601 outputs captured images into the movementfeature extractor 202. The control signal generator 205 generates thecontrol signal 211 using the schemes described in the embodiments 1-3.If it is possible to reduce the processing load of the movement featureextractor 202 by controlling the monitoring camera 1601 using thecontrol signal 211, the monitoring camera 1601 may be controlled usingthe control signal 211. For example, if it is desired to decrease theprocessing frequency of the movement feature extractor 202, thecapturing frequency of the monitoring camera 1601 may be reduced. If itis desired to decrease the number of processed pixels of the movementfeature extractor 202, the resolution of the monitoring camera 1601 maybe reduced. If the control signal 211 instructs to exclude any one ofthe monitoring cameras 1601 from the process, similar effects can beachieved by such as powering off the camera or turning the orientationof the camera.

The video monitoring device 200 according to the embodiment 4 mayinclude a movement feature storage 1602 in addition to the configurationdescribed in the embodiment 1. The movement feature storage 1602 storesmovement features extracted by the movement feature extractor 202. Byadjusting the data size of movement features using the control signal211, it is possible to optimize the storage size for movement features.For example, in addition to the objective processing load 206 (orinstead of it), a data size of movement features stored within a certainperiod is specified. The test pattern generation table 500 or theconfiguration pattern correspondence table 1300 describes, in additionto reducing the processing load (or instead of it), patterns fordecreasing the data size of movement features. The control signalgenerator 205 generates the control signal 211 according to thesevalues, and outputs the control signal 211 into the movement featurestorage 1602. The movement feature storage 1602 reduces, according tothe control signal 211, the data size of movement features using methodssuch as thinning vector data at certain intervals.

Embodiment 5

In an embodiment 5 of the present invention, a method for building thevideo monitoring system 100 described in the embodiments 1-4 will bedescribed in terms of the sequence for determining camera locations.

FIG. 17 is a flowchart showing a sequence for building the videomonitoring system 100. FIG. 17 shows a sequence for determininglocations of monitoring cameras when installing the video monitoringsystem 100 described in the embodiments 1-4 into a facility. Whendetermining the monitoring camera locations, the functions of the videomonitoring system 100 described in the embodiments 1-4 are utilized.Hereinafter, each step in FIG. 17 will be described.

(FIG. 17: steps S1701-S1703)

An administrator building the video monitoring system 100 installsmonitoring cameras in the facility at as many locations as possible(S1701). At this time, it is desirable to cover the entire facility bythe monitoring cameras. Next, a test scene is recorded for a certainperiod using each of the installed monitoring cameras (S1702). Next, aninitial value of the objective processing load 206 is set (S1703). Aprocessing frequency of image extraction process or electric powerconsumption of servers may be specified as the objective processing load206.

(FIG. 17: steps S1704)

The video monitoring system 100 performs the processes (such as movementfeature extraction or simulations) described in the embodiments 1-4using the configured objective processing load 206 and the test scene.The process load calculator 310 calculates a temporal average of theprocessing load for each of the camera images 201 related to the testscene. At the time performing this step for the first time, test scenesof all monitoring cameras are used. As described in the subsequentsteps, the number of monitoring cameras will be reduced as thisflowchart proceeds.

(FIG. 17: steps S1705)

Steps S1706-S1708 described below are performed for test scenes of allmonitoring cameras. An index i is assigned to each of the monitoringcameras. The value of i will be increased as the process loop proceeds.

(FIG. 17: steps S1706)

It is determined whether the average processing load for camera icalculated in step S1704 is at or below a predetermined value. Thisdetermination may be performed by any one of the functional units in thevideo monitoring system 100 or the administrator may determine byvisually checking the numerical values. The determinations in stepsS1707 and S1709 may be performed similarly. If the average processingload is at or below the predetermined value, the flowchart proceeds tostep S1708. Otherwise the flowchart proceeds to step S1707.

(FIG. 17: steps S1707)

It is determined whether the average processing load of the camera icalculated in step S1704 is the minimum among all of the camera images201. If it is the minimum, the flowchart proceeds to step S1708.Otherwise the flowchart proceeds the loop of step S1705.

(FIG. 17: steps S1708)

The camera which is determined in step S1706 or in S1707 that itsaverage processing load is low may not be frequently used. Thus suchcamera is excluded from cameras installed in the facility. The camerasexcluded in this step will also be excluded in the subsequent processloops.

(FIG. 17: steps S1709)

It is determined whether the number of cameras that are not excluded instep S1708 reaches at or below a predetermined value (S1709). If notreached, the objective processing load 206 is decreased from the currentvalue (S1710), and the flowchart returns back to S1704 to repeat thesame process. If the number of cameras reaches at or below thepredetermined value, remaining cameras are installed.

Embodiment 6

FIG. 18 is a functional block diagram of the video monitoring device 200according to an embodiment 6 of the present invention. In the embodiment6, a configuration example will be described where it is assumed thatthe video monitoring system 100 described in the embodiments 1-4operates in a train station. Other configurations are similar to thoseof the embodiments 1-4. Thus hereinafter differences will be mainlydescribed.

As the sensor information 207, ticket gate passing information 1801 ortrain operation information 1802 are inputted into the video monitoringdevice 200. The ticket gate passing information 1801 is informationindicating the number of persons passing the ticket gate within acertain period. The train operation information 1802 is informationindicating the time table on which operations of trains are currentlybased. The video monitoring system 100 includes a ticket gateinformation converter 1803 and a train operation information converter1804.

The ticket gate information converter 1803 converts the ticket gatepassing information 1801 into formats that can be simulated by thesimulator 203. For example, if the ticket gate passing information 1801is information indicating the number of entering and exiting people foreach of ticket gates, the location of each ticket gate and thedirections of entering and exiting are managed by coordinate valuesrespectively. The locations of each ticket gate and the enteringdirection are added to the number information of entering people. Thelocations of each ticket gate and the exiting direction are added to thenumber information of exiting people.

The train operation information converter 1804 converts the trainoperation information 1802 into formats that can be simulated by thesimulator 203. For example, the time when the train arrives at theplatform is estimated from the train operation information 1802, and thenumber of boarding and exiting people is estimated from data such asaverage number of boarding and exiting people at the arrival time. Thelocation of train arrival or the location information of exits are addedto the estimated number data.

In the embodiment 6, a specific example of the sensor information 207 isdescribed assuming that the video monitoring system 100 is installed ina train station. Sensors that detect other physical states may beinstalled according to the operational form of the video monitoringsystem 100, and its detection result may be used as the sensorinformation 207.

The present invention is not limited to the embodiments, and variousmodified examples are included. The embodiments are described in detailto describe the present invention in an easily understood manner, andthe embodiments are not necessarily limited to the embodiments thatinclude all configurations described above. Part of the configuration ofan embodiment can be replaced by the configuration of anotherembodiment. The configuration of an embodiment can be added to theconfiguration of another embodiment. Addition, deletion, and replacementof other configurations are also possible for part of the configurationsof the embodiments.

The configurations, the functions, the processing units, the processingmeans, etc., may be realized by hardware such as by designing part orall of the components by an integrated circuit. A processor mayinterpret and execute programs for realizing the functions to realizethe configurations, the functions, etc., by software. Information, suchas programs, tables, and files, for realizing the functions can bestored in a recording device, such as a memory, a hard disk, and an SSD(Solid State Drive), or in a recording medium, such as an IC card, an SDcard, and a DVD.

DESCRIPTION OF SYMBOLS

-   100 video monitoring system-   200 video monitoring device-   202 movement feature extractor-   203 simulator-   204 simulation result viewer-   205 control signal generator-   208 format converter-   500 test pattern generation table-   1300 configuration pattern correspondence table

What is claimed is:
 1. A video monitoring system comprising: an analyzer that analyzes a movement of a moving object in a video captured by multiple of cameras; a simulator that simulates, according to an analysis result by the analyzer, a flow of the moving object in an area captured by the multiple of cameras; a control signal generator that generates a control signal for switching a processing scheme used by the analyzer when performing the analysis according to a simulation result by the simulator; and a pattern table that describes a combination pattern of processing schemes used by the analyzer when analyzing the video captured by each of the cameras, wherein the combination pattern is configured so that a processing load when the analyzer performs the analysis can be reduced than a processing load when the processing scheme with a highest processing load is used, wherein the control signal generator calculates, from the simulation result, a parameter that is correlated with a processing load when the analyzer acquires the analysis result, specifies the combination pattern corresponding to the simulation result according to the calculated parameter, and generates the control signal corresponding to the specified combination pattern, and wherein the analyzer switches the processing scheme according to the control signal generated by the control signal generator.
 2. The video monitoring system according to claim 1, wherein the simulator outputs a first simulation result acquired by performing the simulation based on the analysis result acquired by the analyzer according to the combination pattern, wherein the simulator outputs a second simulation result acquired by performing the simulation based on the analysis result acquired by the analyzer using a second processing scheme having a processing load of the analyzer which is higher than that defined by the combination pattern, and wherein the control signal generator acquires the first simulation result for multiple of the combination patterns, calculates a matching degree between the first simulation result and the second simulation result as the parameter, and generates the control signal that instructs the analyzer to use the combination pattern corresponding to the first simulation result having a highest one of the matching degree.
 3. The video monitoring system according to claim 2, the video monitoring system further comprising a control signal determinator that determines a control signal to be employed among multiple of the control signals generated by the control signal generator, wherein the control signal generator generates the control signal that instructs to perform the analysis using each of one or more of the combination patterns, wherein the simulator performs the simulation according to the analysis result corresponding to each of the control signals generated by the control signal generator, and wherein the control signal determinator employs a control signal among multiple of the control signals generated by the control signal generator having a highest one of the matching degree.
 4. The video monitoring system according to claim 2, wherein the control signal determinator acquires an objective processing load which is an objective value of the processing load of the analyzer, and wherein the control signal determinator specifies one or more of the control signals that achieves the processing load at or below the objective processing load, and employs a control signal among the specified multiple control signals having a highest one of the matching degree.
 5. The video monitoring system according to claim 1, the video monitoring system further comprising map data that describes map information of the area captured by the multiple cameras, wherein the control signal generator calculates, as the parameter, a numerical value that is a numerical value related to the moving object at a point on the map data and that decreases a processing load when the analyzer acquires the analysis result, and wherein the control signal generator specifies the point where the processing load is predicted to be decreased according to the parameter and the map data, and decreases the processing load at the point according to the prediction.
 6. The video monitoring system according to claim 5, wherein the pattern table describes multiple relationship patterns between the parameter and the combination pattern, wherein the control signal generator acquires an objective processing load which is an objective value of the processing load of the analyzer, and wherein if the processing load does not reach at or below the objective processing load even by generating the control signal using any one of the relationship pattern described in the pattern table, the control signal generator generates the control signal that instructs to cumulatively apply other one of the relationship pattern.
 7. The video monitoring system according to claim 6, wherein if the parameter does not match with any one of the relationship pattern described in the pattern table, the control signal generator generates the control signal that instructs to reduce the processing load for a predetermined one of the camera regardless of the relationship pattern.
 8. The video monitoring system according to claim 3, the control signal generator and the control signal determinator intermittently repeat an operation where the control signal generator generates the control signal and where the control signal determinator determines the control signal to be employed.
 9. The video monitoring system according to claim 1, wherein the control signal generator generates the control signal that instructs the camera to reduce the processing load according to the combination pattern, and wherein the camera reduces the processing load according to the control signal.
 10. The video monitoring system according to claim 1, the video monitoring system further comprising a storage unit that stores the analysis result by the analyzer in a storage device, wherein the control signal generator generates the control signal that instructs the storage unit to reduce a data size of the analysis result according to the combination pattern, and wherein the storage unit reduces the data size of the analysis result according to the control signal.
 11. The video monitoring system according to claim 1, the video monitoring system further comprising a sensor that detects a physical state of the area captured by the camera or of a peripheral area of the area, wherein the simulator performs the simulation using the physical state detected by the sensor.
 12. A video monitoring method comprising: a step of reading a pattern table that describes a combination pattern of processing schemes used when analyzing a movement of a moving object in a video captured by multiple of cameras; an analysis step of analyzing a movement of a moving object in a video captured by each of the multiple cameras; a simulation step of simulating, according to an analysis result in the analysis step, a flow of the moving object in an area captured by the multiple of cameras; and a control signal generation step of generating a control signal for switching a processing scheme used in the analysis step when performing the analysis according to a simulation result in the simulation step, wherein the combination pattern is configured so that a processing load when performing the analysis in the analysis step can be reduced than a processing load when the processing scheme with a highest processing load is used, wherein the control signal generation step calculating, from the simulation result, a parameter that is correlated with a processing load when acquiring the analysis result in the analysis step, specifying the combination pattern corresponding to the simulation result according to the calculated parameter, and generating the control signal corresponding to the specified combination pattern, and wherein the analysis step switching the processing scheme according to the control signal generated in the control signal generation step.
 13. A method for building the video monitoring system according to claim 1, comprising: a step of installing multiple cameras in the region; an analysis step where the analyzer analyzes a movement of a moving object in a video captured by the camera; a simulation step where the simulator simulates a flow of the moving object in the area captured by the multiple cameras according to an analysis result by the analyzer; and a removal step removing, from the area, the camera causing a processing load of the analyzer and the simulator lower than a predetermined value, or the camera causing a lowest processing load of the analyzer and the simulator among the multiple cameras, wherein the analysis step, the simulation step, and the removal step are repeated until a number of the cameras installed in the area reaches at or lower than an objective number of camera. 